SOIT: Segmenting Objects with Instance-Aware Transformers

This paper presents an end-to-end instance segmentation framework, termed SOIT, that Segments Objects with Instance-aware Transformers. Inspired by DETR, our method views as a direct set prediction problem and effectively removes the need for many hand-crafted components like RoI cropping, one-to-many label assignment, non-maximum suppression (NMS). In multiple queries are learned to directly reason of object embeddings semantic category, bounding-box location, pixel-wise mask in parallel under global image context. The class can be easily embedded fixed-length vector. mask, especially, is group parameters construct lightweight instance-aware transformer. Afterward, full-resolution produced transformer without involving any RoI-based operation. Overall, SOIT introduces simple single-stage framework both RoI- NMS-free. Experimental results on MS COCO dataset demonstrate outperforms state-of-the-art approaches significantly. Moreover, joint learning tasks unified query embedding also substantially improve detection performance. Code available at https://github.com/yuxiaodongHRI/SOIT.